Conventional projectors generally need to be positioned at far distances from display surfaces to create an adequately large projection area. And without proper mounting and placement, the light path emitted may not remain obstruction free. Presenters passing in front of or near those display surfaces often face bright projector light, shining into their eyes, or they may be colored with the projection image while adding their shadow to the display surface.
Short-throw projectors help alleviate these problems and are especially necessary for interactive projection systems. These projectors are ideally positioned on a wall or ceiling above the target display surface. The short-throw technology enables the projector to output a sufficiently large screen image onto the wall below, and the high mounting provides an unobstructed screen image which can be approached easily by interactive users. Short-throw projectors, mounted above and close to the target surface, also offer diminished light projection into the eyes of presenters/attendees and reduce shadows caused when users intersect the light output path. For these reasons, many interactive whiteboard systems use short-throw projectors for their display component.
In an interactive electronic whiteboard system, a digital pen replaces ink and provides an input interface to draw content onto a virtual whiteboard. Traditionally the digital pens are physical hardware elements modeled after their ink counterparts to offer a natural user experience. They may transmit their location or are otherwise remotely monitored for their position over a display surface. Buttons, orientations, or other unique identifying characteristics of each pen may also be detected by supporting applications and can be used to affect an operation, modify or manipulate a drawing feature, or change an activity. Computer applications can generate different outputs on the whiteboard surface using such pen interaction. For example, the moving pen position information is monitored by a computer application which in turn paints electronic color pixel elements onto a virtual canvas at the corresponding pen position in an application window. The canvas being simultaneously displayed by the application makes it appear as if the electronic ink was deposited and drawn using the virtual pen. This type of technology is not new and electronic painting applications using tablets, mice as “pen” surrogates, or other inputs sources are quite popular and readily available.
As interactive whiteboard systems have gained in popularity, systems offering support for multiple pen inputs are becoming more desirable. Multiple inputs allow simultaneous users to share in the interaction, or provide an easy way to distinguish differing interactive elements with particular input devices. Unfortunately, multi-input aware application support is limited. Application developers must use system vendor provided APIs for particular hardware or rely on generic operating system support when vendors configure pen inputs to function as mouse devices to maximize application operability. Even the developer APIs in operating systems that allow applications to gain access to multiple devices and their inputs are relatively new or have not been well defined. And, operating systems themselves are typically configured for a single user interaction at a time and treat multiple devices of the same type as a single unified device. Multiple mice, for example, can be attached and recognized by many computers and their operating systems. However, manipulation of each individual mouse generally functions to control the single provided system pointer. The result is that multiple connected mice are unified to control/share a single screen element.
Some low level developer APIs within various operating systems do allow applications access to the individual unique mouse devices. But it is often the application's responsibility to create separate pointers and position functions for each mouse device that may only operate with the application's GUI (graphical user interface). Therefore, applications supporting multiple-pens are usually specialized for a particular purpose or to a particular set of devices. In other cases, applications choose to support only the limited single-device operating system APIs.
Microsoft operating systems provide a low-level Raw Input APIs to developers through which an application can obtain information regarding any human interface device (HID) connected to the system. Microsoft also developed a managed code solution with the MultiPoint Mouse SDK (software development kit) that exposes a common application environment supporting multiple mouse devices. This SDK offers an application a window platform for tracking the multiple devices and includes helpful support for managing cursor icons, position, and other mouse related GUI and tracking events.
Epson's BrightLink 475Wi Interactive Projector, for example, is configured as a short-throw projector with an electronic whiteboard application that creates an interactive whiteboard projector (IWP). This all-in-one product solution seeks to replace the traditional whiteboard by providing dual electronic pens operating over a drawing region output by the projector. The pens are detected and managed by a receiver and provide input to an internal whiteboard application. Providing similar input to any application on a host PC that might be configured to use the IWP and pens is an object of the present invention.
FIG. 1 shows an example of a number of interactive whiteboard projectors (IWPs) 10 connected on a network 12 to a host computer 16. Since the short-throw projector is mounted high on a wall or in the ceiling, a functional interface break-out box 14 extends down from the projector and is positioned next to the display surface 30. This box contains user buttons and the interactive pen devices for the whiteboard applications. As a network based projector, the IWP may be configured to share content and interaction with other IWP devices. Therefore, multiple user pens inputs could be provided and made available within remote supporting applications.
In effect, a network projector can be harnessed to output display content gained from some remote source device. Device inputs on the network projector may too have use by a remote target device.
A key challenge with providing so many input devices is sharing the device hardware with applications on a host PC connected to the IWP. In prior product solutions, the supplied input hardware was physically connected to the host PC where drivers were installed to interpret the pen inputs. However, since the IWP is a network based product, additional cabling for connecting input devices is unreasonable. Transmitting the hardware signals across the existing network is one solution. This could require a network based USB driver for the devices, for example, to be installed on the PC. However, users generally do not like to install device drivers on their systems.
With a network projector, sending application screen content from a host PC across the network to the projector display already requires some transmission method. As discussed in co-pending application Ser. No. 13/332,751, Microsoft's remote desktop services can be ideally suited for this situation in a Windows operating system environment. Sending input devices through this transmission interface is possible. Unfortunately, in a hosted remote desktop session only a single mouse and keyboard device is provided by the remote desktop connection client application. Multiple mice on the client (projector) would be unified into a single mouse device through the remote desktop protocols. Applications running the remote desktop session would only recognize the single mouse device.
Physical connection of additional devices on the host, already demonstrated as undesirable for cabling and driver reasons, is also problematic when operating in a remote desktop session. This is because a host machine's local HID devices are restricted in operation to the input desktop which will be the Windows Logon screen desktop during the remote desktop connection. This means that multiple-HID devices cannot be seen or utilized in applications running within a hosted remote desktop session.
What is desirable is a solution that allows the IWP and interface box client to connect with a host PC using remote desktop services to gain the display output performance expected for desktop applications. Further, the desirable solution would also allow the multiple pen devices on the IWP and interface box client to be available to any multi-input applications launched on the host PC within the remote desktop session. Ideally this would not require extensive additional software or driver installation. Additionally, it would be desirable to share pen devices of a IWP with applications on a host PC when the IWP display is not being utilized by the same PC. The present invention is directed to achieving these and other objectives.